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ABSTRACT 


This research studies the positive and negative perceptions of graduate students from the United 
States studying issues of sustainable electronics and electronics manufacturing in India as part of 
a National Science Foundation Integrative Graduate Education and Research Traineeship (IGERT) 
curriculum. The purpose of this paper is to discuss the use of an online photoelicitation method, 
which intends to probe learning experiences from the graduate students’ international experience 
through the use of participant-generated photographs taken during the trip. This research provides 
insight on the importance of international learning experiences in graduate-level engineering educa¬ 
tion and the value of non-traditional formats of graduate student learning. This research exemplifies 
the use of this novel method in research and educational assessment of non-traditional learning 
environments, as well as offers interested engineering educators guidelines to implementing this 
technique in their individual contexts. 

Keywords: Photoelicitation Methods; Electronics Manufacturing; International Engineering Education 


INTRODUCTION 


For engineering educators, deep engineering knowledge and learning can often not be assessed 
through quizzes, exams, or problem sets. This paper may offer another tool in the innovative engi¬ 
neering educator’s assessment “toolbox” for extracurricular, project-based learning, or international 
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experiences, to name a few. Photoelicitation is a research method that uses photographs in order 
to elicit deeper reflections or responses from research participants. Used in educational settings, 
photoelicitation can also be a novel resource for assessment of deep understanding and student 
values or perceptions. In this paper, we show the use of photoelicitation methods as insight into 
graduate student learning in an international context—a context that is complex, extracurricular, and 
cannot be assessed through tests. Rather, using multimedia tools, photographs taken during the 
experience, and online survey software, deep student reflections were elicited which yield interest¬ 
ing research findings and evidence of graduate student professional development and awareness. 

The Integrative Graduate Education and Research Traineeship on Sustainable Electronics (IGERT- 
SE) is an NSF-funded collaboration between Purdue University and Tuskegee University. The program 
seeks to provide graduate students with broad interdisciplinary research and professional develop¬ 
ment skills to augment their depth of knowledge in sustainable electronics research. Sustainable 
electronics research seeks to develop of new electronics and manufacturing process in order to 
protect the environment, resources, and health as much as possible in an industry that is often 
known for hazardous materials and manufacturing processes. In addition to a weekly IGERT class 
and professional development meetings, this traineeship included an international component of 
learning, where the trainees and associated professors participated in a two-week trip to India to 
experience international culture and observe attitudes, and viewpoints of electronics manufacturers. 

This study intends to address the following research questions: 

• What are the positive and negative aspects of sustainable electronics as perceived by the 
IGERT-SE students from their experiences and photographs from the trip? 

• How can photoelicitation be adapted to an online survey in order to elicit deep participant 
reflection on a non-traditional learning experience? 

After answering these two research questions, guidance is given for other engineering educators 
who are interested in employing photoelicitation techniques in their research or teaching practices. 
Our study exemplifies the use of multimedia photoelicitation methods through online survey soft¬ 
ware as both a research tool and an educational assessment tool. 


LITERATURE REVIEW 


Global engineering competencies are becoming important in the education of undergraduate 
and graduate-level engineering students, since many engineering teams are comprised of inter¬ 
disciplinary professionals from around the world in global companies, and since many engineered 
products are manufactured at global facilities. Many reports call for the inclusion of international 
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and global competencies for engineers, calling these competencies as necessary for professional 
engineers of the future (International Engineering Alliance 2013; Nerad and Blumenfield 2011; The 
Council of Graduate Schools 2012). Therefore, training in interdisciplinary settings are of interest 
to engineering education researchers. National Science Foundation-funded programs such as In¬ 
tegrative Graduate Education and Research Traineeship (IGERT) collaborations, along with other 
doctoral and post-doctoral engineering education researchers have sought to provide students 
with international engineering experiences, as well as document the advances in global competen¬ 
cies learned by students. Reports from IGERT international experiences have been documented in 
literature (Cutler and Borrego 2010; Gamse, Espinosa, and Roy 2013), and their gains are consistent 
with the recommendations from other reports, indicating that students make advances in research 
ethics (The Council of Graduate Schools 2012), understanding the importance of inter-cultural com¬ 
munication skills (Oladiran et al. 2011; Blumenfield and Nerad 2012; Nerad and Blumenfield 2011), 
and learning to work with people who define problems differently (Downey et al. 2006). The as¬ 
sessment of these global competencies is not clearly defined. General frameworks exist which try 
to map intercultural competencies, assess global programs, and include international instruction 
based on learning outcomes for students (Blumenfield and Nerad 2012; Nerad and Blumenfield 2011; 
Deardorff 2006; Lohmann, Rollins, and Joseph Hoey 2006; Rubin and Matthews 2013; Sadrozinski 
2008), but few studies specifically research graduate level international education. 

In order to assess the reflections of graduate engineering students in their international experi¬ 
ences in India through a different IGERT collaboration, the researchers in this study used photoelici¬ 
tation methods, which employ photographs or pictures as prompts in interview protocol in order to 
better facilitate interview dialogue and reflection from participants. Photoelicitation as an interview 
technique has been used across disciplines for decades, spanning the disciplines of anthropology and 
sociology (Tunnell 2012; Schwartz 1989; Cappello 2005), psychology (Heisley and Levy 1991; Keats 
2010), education (Schulze 2007; Wright and Larsen 2010; Carlsson 2001; Preskill 1995; Thompson 
and Gunter 2007), and other social sciences (Clark-lbanez 2004) in order to better facilitate inter¬ 
view dialogue and reflection from participants through the use of photographs or pictures (Harper 
2002). These images can be provided by the researcher or provided by the participant. Auto-driven 
(participant-generated) photoelicitation uses images drawn or photographed by the participant, 
from the participant’s point of view, to elicit a more complete understanding of meaning, purpose, 
and reflection through the interview process (Heisley and Levy 1991). Auto-driven photoelicitation 
has achieved success as a method, especially when working with children (Epstein, Stevens, and 
Mckeever 2006; Cappello 2005), marginalized populations (Harley 2012), and in participant action 
research (Harley 2012; Jenkings, Woodward, and Winter 2008). Previous studies using photoelicitation 
techniques are distinctly defined as researcher-generated photoelicitation or participant-generated 
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photoelicitation. Literature on photoelicitation analyses and interview techniques verify this line and 
appropriate the use of each technique for different circumstances (Guillemin and Drew 2010). The 
number of studies using photoelicitation as an interview technique are numerous and can be found 
in many complete literature reviews available (Pain 2012; Harper 2002). 

The use of photoelicitation as a method also has theoretical and philosophical connotations. For 
example, the benefits of photoelicitation from literature include the establishing of trust and rap¬ 
port between participants and researchers, increasing interview dialogue and depth of reflection, 
and reducing the power gap between interviewer and participant, as the participant is perceived as 
the expert in conversation (Pain 2012). Most importantly, guidance of interview using participant¬ 
generated photographs allows the participant to place importance and reflect on issues that the 
researcher, as an outsider, might not initially perceive to be important. This idea, called photovoice, 
makes the assumption that participants “can best identify and represent their own realities” (Harley 
2012, p. 322). 

In the field of engineering education research, photoelicitation has been employed by research¬ 
ers in order to understand cross-disciplinary identity development (Forin, Adams, and Hatten 2012; 
Hatten, Forin, and Adams 2013), to better elicit deep interview responses while studying gender in 
engineering (Morley et al. 2011), and to study the use of photoelicitation as a pedagogical method in 
addition to a research method (Jordan et al. 2009). Each of these studies were conducted through 
interview situations of different sorts, where participants were asked to either bring or email the 
researchers several pictures related to the scope of the study. 


METHODS 


Nine graduate students from Purdue University and Tuskegee University Materials Science and 
Engineering departments and related engineering disciplines, such as Mechanical Engineering, were 
involved in the traineeship and participated in the IGERT-SE India trip. The trip was composed of 
tours to several electronics manufacturers (both Indian companies and international companies) as 
well as several Non-Governmental Organizations in northern India. Seven students completed the 
photoelicitation survey. All of the participants in this study are current graduate students studying 
materials science and engineering. Since the survey was anonymous, gender data was not correlated 
with the responses of the students through the online survey. Although faculty attended the trip as 
well, this study focuses on student learning. Of the students that responded to the online survey, only 
one participant identified never having travelled internationally before this experience. This paper 
will discuss the results from the photoelicitation survey and the benefits of using photoelicitation in 
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an online format for data collection. The research protocol and consent forms for the photoelicita¬ 
tion survey were approved through the university’s Institutional Review Board, and the participants 
were informed of the purpose of this photoelicitation study to understand their perceptions and 
international experience with sustainable electronics in India during Summer 2013. Students chose to 
participate in an online photoelicitation survey regarding their perceptions. Photographs submitted 
by the students were taken personally, not as part of the study. Cameras on personal cellular phones 
and personal digital cameras were used to capture the images. No limits were placed on the number 
of photographs that could be taken by students. However, certain sites banned photography, result¬ 
ing in no photographic documentation from those experiences. No IRB approval or consent forms 
were given to the subjects of the photographs: For this reason, we have blurred out the identifying 
features of the subjects of the photographs presented in the results section. To our knowledge, the 
only manipulation of the photographs occurred when this blurring was done. 

The following indicate the differences between the current study and previously documented 
photoelicitation techniques: 

1. This study was conducted after the trip to India: The photographs generated by the students 
were already taken during the trip without the intent to capture or reflect on sustainable elec¬ 
tronics in India. 

2. The photographs that were elicited from the students in the first part of the study were not 
necessarily taken by each student. One participant, an avid photographer, took most of the 
photographs from the trip, and these photographs were available to the students to submit 
for photoelicitation. Students submitted photographs that reflect aspects of sustainable elec¬ 
tronics in India accompanied by a description and an interpretation on the connotation of the 
picture. The written answers were coded by the researchers. 

3. This research is trying to capture the perception of the outsider (here, graduate students from 
the United States that research aspects of sustainable electronics) in a foreign culture. Most 
photoelicitation studies are interested in studying the native perception within a culture of 
issues to reflect and solve social issues (Jenkings, Woodward, and Winter 2008; Thupayagale- 
Tshweneagae and Mokomane 2013; Van Auken, Frisvoll, and Stewart 2010; Slutskaya, Simpson, 
and Hughes 2012) or are interested in visitor perceptions for tourism purposes (Westwood 
2006; Matteucci 2013). 

The survey, conducted through online survey software, included a total of 29 questions, includ¬ 
ing brief demographic data and confirmation that the photographs provided were a product of the 
India trip in June 2013. The remainder of the survey systematically asked students to upload and 
describe photographs from their trip that represented the most-positive, the second-most posi¬ 
tive, and the third-most positive aspects, and most-negative, second-most negative, and third-most 
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negative aspects of electronics manufacturing in India. Following each of the photographs uploads, 
students were asked to describe the photo they uploaded, and then answer the question “Why 
does this picture have a positive/negative connotation?” The written survey results describing the 
participant-generated pictures were open-coded in order to determine primary and (if applicable) 
secondary and tertiary themes. 


POSITIVE PHOTOELICITATION RESULTS 


The participant-generated photographs that represented positive aspects of electronics in India 
are presented in this section. Since each of the seven survey participants provided three “posi¬ 
tive” pictures, a total of 21 "positive” pictures were collected. Of these, only two were duplicated, 
resulting in a total of 19 unique participant generated photographs. The primary, secondary, and 
tertiary codes from the written descriptions of the photographs were coded into six emergent 
themes: (1) Safety, Cleanliness, and Working Conditions; (2) Community Improvement; (3) Em¬ 
ployee Education; (4) Sustainability; (5) Women’s Equality; and (6) Student Experience. Figure 1 
shows the frequencies of each of these themes in the student responses. These six predominant 
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themes were reflected in the descriptions that students provided when they uploaded the pho¬ 
tographs to the online survey. 

The number of times that safety, cleanliness, and good working conditions were mentioned is 
much higher than the frequencies by the other themes. Students expressed appreciation for worker 
safeguards, safety education, and guest/visitor safety regulations, as well as the presence and use 
of appropriate personal protective equipment. For the “most positive” picture shown in Figure 2, 
the prompt was answered: 

“This picture has a positive connotation due to the cleanliness and safety implementations 
that are visible. The workers are all wearing proper personal protective equipment (lab 
coats, closed-toed shoes, etc.) and the path for visitor safety is clearly marked in yellow 
tape. There is little to no clutter or miscellaneous articles within the facility.” 

The student who submitted this described the photo: "This is an image on a factory floor. This specific 
facility deposits solder paste onto circuit boards using screen printing, attaches components and heats 
the system to create electrical and mechanical connections between the board and components.” 



Figure 2. Example of a “most positive” picture representing safety measures in 
electronics manufacturing facilities. 
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Specific notation of personal protective equipment and safety regulations may be due to other factory 
tours where such high standards were not upheld, resulting in a particularly “outstanding” connotation 
to the student in retrospect. Another student, in a “third-most positive” photograph, depicted a worker 
demonstrating a coating process, wearing “protective respiratory equipment, which was a rare find.” 

In the frequency analyses of the predominant themes identified in student descriptions, the themes of 
community improvement, employee education, and sustainability were nearly equally represented. The 
picture in Figure 3 was taken during a meeting with students and faculty from the United States and a 
Non-Governmental Organization (NGO) in India, where “they described their missions and goals as an 
organization, their past achievements, as well as the future priorities with respect to sustainable electronics.” 

The student description of the photograph showed how a simple picture of a meeting, for example, 
can represent such a broad and important theme in student learning and global engineering topics. 
However, as this student notes, the positive aspects of sustainability are tainted with the reality that 
regulations are more easily made than enforced: 

“[T]here are efforts associated with improving the status quo of electronics waste and 
handling in India. One of the major concerns of [an NGO near New Delhi] is to reduce 



Figure 3. Example of a “second-most positive" photograph representing non¬ 
governmental organizations’ sustainability efforts in electronics manufacturing. 
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the amount of mercury exposure for Indian citizens. They work tirelessly to create new 
regulations against the use of mercury in manufacturing. The outlook, however, is not as 
positive as it could be, since enforcement in India is not as stringent as the actual regulation.” 

Many students’ descriptions of sustainability-oriented photographs specifically acknowledged 
some facilities’ efforts to manage waste and hazardous materials safely, with concern for the envi¬ 
ronment and the health of the surrounding community. This is demonstrated by the picture shown 
in Figure 4, which describes “a waste handling site for unlicensed recyclers or rag pickers in New 
Delhi. Throughout large Indian cities, one would find large waste sites directly off main roads.” 

Although an outside observer of this picture might think this dump site would have a negative 
connotation, the student submitting this photograph showed that the context and interpretation 
of her or his learning experience through photographs leads to a better understanding of Indian 
culture, community, as well as manufacturing: 

“Though everything about this picture is not positive, the main premise of what’s happening 
here is. We learned that the national average of recycling in India is over 60%, far exceeding 
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the international average of about 35%. The reason why reuse and recycling are so heavily 
emphasized is partly because of the tradition of waste pickers. Many people make a living 
off of sorting through city waste and making use of materials and entire products.” 


The importance of community in many of the student responses through the use of many differ¬ 
ent types of photographs was noted, both in community improvement through sustainability, as in 
the example above, but also in community improvement through education. The issue of workforce 
education was specifically addressed by many students. Several students submitted photographs of 
worker education bulletin boards, where “the company is providing basic materials and properties 
information in order to educate their workers,” since “[m]any of the workers in these factories do 
not have a background that provides them with the education to understand the fundamental sci¬ 
ence behind what they are doing [in their work].” In addition to technical worker education, many 
students noted examples of the combination of education and community improvement. Figure 5 
shows one student’s “most positive” picture. 

The description given by the student submitting the paper: “In the photograph, you can see women 
from impoverished situations learning how to operate and maintain photovoltaic assemblies. The 
students represent many nations from across the globe, including India, Cambodia, [and] Panama.” 
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The prompt asking the student to describe why the picture was rated as most positive yielded a 
response noting the themes of sustainability, community improvement, and education: 

“The photograph truly exhibits the best of instituting sustainable practices. The goal 
of the program depicted here [...] is for each student to return to her home village and 
install as well as maintain solar panels. For many of these villages, beforehand there 
existed no electricity. Installing solar panels will have a considerable effect on the local 
societal structure of each village. Also, it gives each village the opportunity to create a 
local economy, in which they can sell energy or services to neighboring villages. Overall, 
the program hits sustainability in all three of its pillars: environmental with a renewable 
energy source, societal with the addition of electricity to villages and economic since the 
village can sell its energy. On a personal note, this photograph captures the most incredible 
moment of my trip [...I spoke] with a woman from Panama. Using my broken Spanish, I ask 
her how she feels about working here [...]. She responded in Spanish, “We are not working. 

We are studying.” A truly life-changing conversation!” 

Interestingly, this student did not express in her or his description or interpretation the theme 
of women’s equality, which was expressed by other students through other elicited responses and 
photographs, such as these responses to similar pictures of women working in electronics manu¬ 
facturing: “The amount of working women we saw while touring electronic manufacturing facilities 
was shockingly low. This picture has a positive connotation because the ratio of working women at 
this facility was visibly greater than any other we visited.” Another student noted that “[w]omen 
are traditionally housewives in India. Being willing to hire large numbers of women greatly increases 
their control over their own life.” Since engineering tends to be male-dominated, the fact that these 
graduate students were perceptive of women’s representation and equality issues shows a respect 
and appreciation for diversity in technical fields, as well as a value for discussion of justice and 
equality issues in an engineering context. 

“Ranking” Positive Photographs 

One might argue that asking participants to upload simply three positive pictures may have 
elicited similar emergent themes, however, the researchers found that where a participant 
“ranked” her or his photographs, with descriptions and justification of the ranking, researchers 
could identify the frequencies of the themes as a function of the “rank” of the photograph. Fig¬ 
ure 6 shows the distribution of the themes for the most, second-most, and third-most positive 
photographs. 
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Conditions 

Predominant Theme 


- Most positive photo 
-Second-most positive photo 
-Third-most positive photo 


Figure 6. Distribution of theme frequencies for most, second-most, and third-most 
positive photographs generated by students. 


For the most-positive photograph, it is seen that most emphasis is placed on the presence of 
safety measures for employees, cleanliness, and healthy working conditions, while sustainability and 
education were higher for the second-most positive photographs. Education was so highly noted by 
the participants in the “top” two ranks, that it was not mentioned at ail in the third-ranked position, 
which means it was very important to the participants. 


NEGATIVE PHOTOELICITATION RESULTS 


The participant-generated photographs that represented negative aspects of electronics in India 
are presented in this section. Since each of the seven survey participants provided three “positive” 
pictures, a total of 21 “negative” pictures were collected. Of these, only two were duplicated, result¬ 
ing in a total of 19 unique participant generated photographs. The primary, secondary, and tertiary 
codes from the written descriptions of the photographs were sorted into five themes: (1) Hazardous 
Working Conditions; (2) Lack of Personal Protective Equipment; (3) Negative Impact on Community 
Sanitation and Environment; (4) Materials Handling Procedures; and (5) Other. The “other” category 
tended to include student discussions of city traffic and mention of potential hiring of minors. Figure 7 
shows the frequencies of each of these themes in the student responses. 
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Figure 7. Distribution of themes from negative participant-generated photographs. 


The topic of hazardous working conditions was the most frequently referred to in student descrip¬ 
tions of their uploaded photographs. This category included participant reference to lack of safety 
measures, machine safeguards, and otherwise dangerous working conditions, as depicted by Figure 8. 

The description provided by the student: “Similar to other images provided there is a large safety 
issue present in this image. The drilling process is creating sparks and the operator has used paper 
towels to fill in an area that needed support. This is a fire hazard.” 

Other students identified the lack of machine safeguards on other machines in their uploaded 
photographs. For example, one picture of a machine operator was uploaded with the description, 
“This shows a worker operating a complex pneumatic machine with many pinch points and no 
guards...It would take very little effort to drastically lower the risk of this worker losing a finger.” 
One student uploaded a picture of a safety sign as a negative picture. Although an outsider may 
consider signs posted about safety to be more positive than negative, the student reflected on the 
photo that “The fact that we saw more notices about protecting the product produced than safety 
warnings for the workers was very troublesome.” 

Similarly, the category showing student identification of the lack of personal protective equip¬ 
ment is another high-ranking category—so much so that the researchers decided to separate the 
category of protective equipment from general hazardous working conditions in this analysis. Figure 9 
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Figure 8. Example of a “third-most negative" photograph depicting an unsafe drilling 
process with resulting sparks. 



Figure 9. Example of a “most negative" picture showing a lack of appropriate personal 
protective equipment. 
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shows one example of a student’s submission of a “most negative picture” regarding the lack of 
personal protective equipment within an electronics manufacturing plant. 

The description given by the student: “This is a very common situation in electronics manufac¬ 
turing. The operator is running a coating machine with open vats of chemicals. He has an ill-fitting 
mask which is not rated to protect against organic solvents and no safety glasses and cloth gloves 
which will just hold spilled chemicals on his skin.” 

Many of the students that uploaded pictures of processes or workers mentioned these types of 
glaring safety errors on the part of the factory toward its workers. One student uploaded a picture 
which showed “two IGERT students putting on ear protection in an extremely loud manufacturing 
facility. We had to ask for the hearing protection which they had on site but no one used.” Another 
student similarly noted that in a lead acid battery manufacturing facility the students toured, issues 
regarding improper regulation and handling of materials caused significant risk of worker and visitor 
hazards as well as harmful environmental impact. 

“Visitors were exposed to sulfuric acid fumes, lead oxide exposure and open heating 
elements. The working conditions were extremely unsafe and allowed for significant 
contamination of the environment... Improper regulation or lack of enforcement of 
regulation of the lead acid battery industry encouraged significant environmental damage.” 

The lead acid battery manufacturer came up several times in the responses of different students; 
however, no students had personal pictures from this factory because photography was prohibited. 
Instead, students uploaded similar pictures, but wrote caveats into their descriptions of the pho¬ 
tographs, for example: “It should be noted that if I had a picture from the lead acid battery manu¬ 
facturer that would be the most negative picture.” Some students similarly used pictures they had 
in order to guide a response for photographs they wished they could have taken. For example, the 
same picture presented before in Figure 9: 

“[...] reminds me of facilities we saw which were much worse that we were not allowed 
to take pictures in. Imagine that machine only in a half burned down building covered in 
pigeon poop, pouring molten lead and shooting out blasts of open flames. It is painful to 
think about the lives of the workers in the worst facilities in India.” 

The improper use, handling, and storage of hazardous materials also were present themes in 
student photographs and responses. Figure 10 shows just one student example of a picture of open 
materials, with little or no safeguards present. The student describes the safeguards for this “solder 
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bath used to fuse components” as “almost laughable. There is nothing protecting workers from the 
high temperatures or the fumes.” 

Overall, student emphasis on the lack of safety measures and often-absent personal protective 
equipment for workers and visitors alike demonstrates student understanding of some of the ethical 
issues surrounding electronics manufacturing in countries with fewer regulations and lower enforce¬ 
ment of health and safety standards. 

“Ranking” Negative Photographs 

Just as conducted for the “positive” photographs in the previous section, Figure 11 shows a plot 
of the frequency of each of the categories for each student’s “negative” pictures. 

One interesting result of this data is that, although the frequencies for safety and personal 
protective equipment categories are the highest ranked overall, most students mentioned ele¬ 
ments of their negative pictures having a negative effect on community sanitation and the envi¬ 
ronment in their descriptions and reflections. The second-most negative pictures, as chosen by 
the participants, overwhelmingly focused on dangerous working conditions and lack of protective 
equipment, whereas the third-most negative pictures acted as “catch-alls”, where students felt 
the need to continue to show negative examples of safety violations, but were also compelled to 
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-Most negative picture 
-Second-most negative picture 
Tljird-most negative picture 


and Environment 

Predominant Themes 

Figure 11. Distribution of theme frequencies for most, second-most, and third-most 
negative photographs generated by students. 


explain some other components that fell into the “other” category. One student mentioned she or 
he had overheard rumors of minors working on an assembly line, and that was why a photograph 
of a crowded production floor was chosen as the third-most negative picture—not so negative 
to be a "most negative” picture, but important enough to the student learning experience that it 
justified a related picture. 


DISCUSSION 


What are the positive and negative aspects of electronics manufacturing as perceived by the 
ICERT-SE students from their experiences and photographs from the trip? 

Specific answers to this question have been discussed at length in the results section based on the 
emergent themes and discussion from the students, frequency analysis of the themes, and the rank¬ 
ing of the photographs by the students. It is important to restate that the students were not aware 
that this photoelicitation study would be done when they travelled to India and participated in the 
international learning experience. The photographs taken during the trip by any of the participants 
could be used by any of the other participants during the survey in order to demonstrate concepts 
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the students thought were positive and negative. Although the pictures elicited may potentially have 
changed slightly if the students were instructed to take pictures representing particularly positive 
and negative aspects of the electronics manufacturing processes during their experience, based 
on the level of reflection provided by the participants for each of the photographs, especially the 
mention of experiences that were not captured by photographs, it is likely that the overall themes 
would have remained similar to the results captured here. It is also important to remember that 
the data reported here are graduate students’ perspectives on issues in sustainable electronics 
manufacturing practices. Many of the prior understandings of these students have been informed 
through U.S. standards and practices in sustainable electronics research. 

How can photoelicitation be adapted to an online survey in order to elicit deep participant reflec¬ 
tion on a non-traditional learning experience? 

In this study, an auto-driven (participant-generated) photoelicitation experience was employed 
by asking students to upload pictures in retrospect of participating in an international learning ex¬ 
perience related to electronics research. This is a unique method that has not been applied before 
in photoelicitation research. By asking students to upload photographs taken on the trip to be as¬ 
signed into the categories of most positive, second-most positive, third-most positive, most negative, 
second-most negative, and third-most negative, a wealth of information was gained through the 
descriptions and interpretations of the photographs. Sometimes, the interpretation was counter¬ 
intuitive to what an “outsider” might assume about the photograph, calling to mind the “positive” 
picture of the roadside waste that was submitted to indicate a higher national recycling rate due 
to waste pickers (Figure 4). Additionally, since approximately six months passed between the trip 
and the survey, the themes emerging from this study represented long-lasting lessons learned 
from the trip. In a few instances, one student uploaded one picture to reflect a positive image of 
electronics manufacturing, and a second student uploaded the same picture to reflect negative 
connotations instead. This, rather than being a contradiction of data, shows the necessity of a 
participant-generated photoelicitation experience, with as few “leads” or “prompts” as possible, in 
order that the participants can reflect on their own constructed learning experiences independently. 
Additionally, by asking students to provide photographs from their trip, the researchers learn what 
themes were most important to students, without generating researcher bias that may have come 
from researcher-generated photographs at the onset of the study. 

Advantages and Limitations of Online Photoelicitation and the Study 

The online format of the survey may have both positive and negative effects for participant-gen¬ 
erated photoelicitation methods. Jordan et al (2009) identifies several advantages as “affordances” 


18 


SUMMER 2015 





ADVANCES IN ENGINEERING EDUCATION 

Research and Assessment of Learning Environments through Photoelicitation: 
Graduate Student Perceptions of Electronics Manufacturing in India 



of photoelicitation as a research methodology and a pedagogical technique, which include accom¬ 
modation of diverse background, the use of technology, and elicitation of multiple perspectives, as 
well as the opportunity for student learning through deep reflection on experiences (Jordan et al. 
2009). We find similar affordances and advantages. By using an online format for the photoelicita¬ 
tion survey, students could take the time to find the picture that best represented their experiences, 
rather than being rushed by researcher-led protocol, as well as using photographs from their own 
files. Based on the methods shown in this study, this novel online method for participant-driven 
photoelicitation should be considered for other researchers considering probing learning experi¬ 
ences of a reflective and mature audience. One potential drawback of employing an online survey in 
this manner is that reflections require mental energy and time, and it is more difficult to type a story 
than tell a story to a listener. However, the online descriptions excelled in quality: word choice and 
student meaning was generally direct, clear, and persuasive. The limitations of this specific study 
include the small number of students that were involved in this trip, as well as the participants’ U.S.- 
centric views on electronics manufacturing and engineering processes which introduce bias in the 
interpretation of the situations and photographs. 

Guidance for Engineering Educators Employing Photoelicitation Research or Evaluation Methods 

Photoelicitation can be used as a method for research, a method for evaluation and assessment, 
or both. In this research, we have shown how photoelicitation (which, up until this point had only 
been used in face-to-face interviews) can also be used in an online survey format. Additionally, in 
this work, we extend the use of photoelicitation from a purely research method to an assessment 
method. 

Ontological and Epistemological Considerations 

Guidance for engineering educators interested in using this in their educational settings stems 
from ontological and epistemological points of view: This is a qualitative method, so rather than 
searching for a positivist overarching “answer” or “truth” about this kind of learning, we are looking 
to probe each individual’s unique truth (i.e. what each student experienced.) Even though the stu¬ 
dents experienced the same meetings and tours on the trip, the knowledge gained by each graduate 
student on the international trip is fundamentally different, since knowledge is not universal, and 
depends on students’ prior experiences with international issues, views on sustainability, ideas on 
gender issues, and the like. We also approach this as an evaluation method with these ontological 
ideas in mind: If each individual has her or his own truth, then a method to assess individual learning 
should not be binary, with a “right” or “wrong” answer, or even a variety of pre-determined choices. 
This is one reason why open-ended responses and student selection of their own photographs in the 
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positive or negative experiences were appropriate for this study. It is also the reason why sometimes 
the same picture was placed in the positive category by one student and the negative category by 
another student, which is enlightening for educators and researchers alike. Additionally, assessment 
in our case is not equivalent with “scoring” in terms of achieving a grade. Graduate students in this 
international experience were a unique population: The experience was extracurricular, so there were 
not grades associated with the responses. If there had been, likely, students would have anticipated 
this and changed their reflections to be aligned with what they thought the professor wanted to 
hear. This is ineffective and counterproductive because it would elicit inauthentic responses and 
would ruin the research integrity of the assessment. 

Learning Environment Considerations 

It is important to determine if the learning environment is appropriate for the use of qualitative 
photoelicitation techniques. It might be the case that the skill/learning is better assessed through 
a written reflection or traditional exam, rather than through this method. This method is great for 
non-traditional educational settings, small groups of students, experiential learning, and interna¬ 
tional learning. It will be of great interest to the authors to note how other engineering educators 
use and adapt this method. 

Online Survey Software Considerations 

Online methods offer the advantage of avoiding transcription and coding of spoken words. It 
is good practice to double-check that the online survey software used allows the survey-taker to 
upload pictures into the survey, as well as to provide written reflection and any other types of ques¬ 
tions that will be incorporated. This study employed Qualtrics software, which worked well. Also, 
upfront planning about data representation and analysis aids in efficient planning and survey data 
collection processes. 

Photographic Considerations 

Determine if participant-generated photographs or instructor/researcher-provided photographs 
will be used. In this study, we used participant-generated photographs because we felt they were 
more authentic. The students had shared all their photographs with each other, so they had access 
to the same photographs from the trip. However, if the assessment is for students to go out with 
the assignment to take pictures of a certain phenomenon and then come back and describe it in 
an assessment, for example, the circumstances will be different. On this note, make sure that all 
students have access to a smartphone or camera: Even in this digital age, students may not have 
access to digital photography equipment, or may feel uncomfortable saying they cannot afford such 
technology if everyone else is using the newest available smartphone technology. 
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Data Usage and Sensitivity Considerations 

If you are intending to publish your results, Institutional Review Board (IRB) approval is still neces¬ 
sary for both survey and photoelicitation methods. Data collected is personal and should be treated 
sensitively. Additionally, if you or your learners are taking pictures of people, the researcher has 
an obligation to either blur out identifying features and/or gain consent before taking the picture. 


CONCLUSION 


The research presented in this study showed the use of an online, participant-generated photoe¬ 
licitation survey to study graduate student learning experiences in an international setting. Written 
responses were open-coded in order to categorize responses into emergent themes for both the 
positive and negative photographs. The photographs themselves are also artifacts for a wealth of 
future planned photoelicitation research on global engineering learning and international experi¬ 
ences. The results of this method for photoelicitation showed promising depth of interpretation by 
the researchers, and generated insightful findings for the field of electronics manufacturing and 
materials science/engineering. Future work using photoelicitation in engineering education could 
relate to other cultural studies in other countries, as students experience engineering norms and 
practices from a global community. Additionally, capturing engineering learning perspectives from 
students even in the United States could afford researchers a rich data set about the engineering 
values that students hold at various stages of their university career and career as a practicing en¬ 
gineer, showing development of a student into the engineering community. 
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